Skip to content

Comments

Track external accumulators in tracer instead of using SparkInfo values#10553

Draft
charlesmyu wants to merge 1 commit intomasterfrom
charles.yu/djm-0000/fix-spark-plan-metrics
Draft

Track external accumulators in tracer instead of using SparkInfo values#10553
charlesmyu wants to merge 1 commit intomasterfrom
charles.yu/djm-0000/fix-spark-plan-metrics

Conversation

@charlesmyu
Copy link
Contributor

@charlesmyu charlesmyu commented Feb 9, 2026 •
edited
Loading

What Does This Do

Updates the metrics in the _dd.spark.sql_plan meta field to use distributions calculated from individual task metrics, rather than the naively summed metrics provided by the StageInfo objects from Spark. This is because StageInfo naively sums all accumulators, even though that may not make sense for certain Spark SQL metrics (e.g. avg hash probes per key for aggr operations). Instead, we should accumulate those ourselves into distribution metrics and emit them accordingly.

Currently in the UI, this is only used in one place (in the Spark SQL metrics in the DJM product), so we're not too worried about changing the format here. UI update to follow.

Motivation

We'd like accurate metrics for Spark SQL operations that can reflect task-level characteristics as a distribution. This brings us more in line with what is shown in the Spark UI:
image

Additional Notes

We can't get rid of the original map that tracks accumulators to stages as we still use that to associate Spark SQL operations to stages. However, we can avoid storing the entire accumulator now, and instead just store a simple map of accumulator ID to stage ID. This will be done in a followup PR: #10645

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@pr-commenter
Copy link

pr-commenter bot commented Feb 9, 2026 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master charles.yu/djm-0000/fix-spark-plan-metrics
git_commit_date 1771530866 1771883443
git_commit_sha af8b844 89df516
release_version 1.60.0-SNAPSHOT~af8b84438c 1.60.0-SNAPSHOT~89df516e40
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1771885530 1771885530
ci_job_id 1449610688 1449610688
ci_pipeline_id 98402425 98402425
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-s2dhypu9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-s2dhypu9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1066951
Total [baseline] (8.762 s) : 0, 8762474
Agent [candidate] (1.072 s) : 0, 1071647
Total [candidate] (8.741 s) : 0, 8740901
section iast
Agent [baseline] (1.23 s) : 0, 1230163
Total [baseline] (9.343 s) : 0, 9342596
Agent [candidate] (1.228 s) : 0, 1228426
Total [candidate] (9.409 s) : 0, 9408732
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent iast 1.23 s 163.212 ms (15.3%)
Total tracing 8.762 s -
Total iast 9.343 s 580.122 ms (6.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.072 s -
Agent iast 1.228 s 156.779 ms (14.6%)
Total tracing 8.741 s -
Total iast 9.409 s 667.831 ms (7.6%) 
gantt
    title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.228 ms) : 0, 1228
BytebuddyAgent [baseline] (629.258 ms) : 0, 629258
BytebuddyAgent [candidate] (631.409 ms) : 0, 631409
AgentMeter [baseline] (29.181 ms) : 0, 29181
AgentMeter [candidate] (29.29 ms) : 0, 29290
GlobalTracer [baseline] (258.085 ms) : 0, 258085
GlobalTracer [candidate] (258.941 ms) : 0, 258941
AppSec [baseline] (33.045 ms) : 0, 33045
AppSec [candidate] (33.313 ms) : 0, 33313
Debugger [baseline] (63.247 ms) : 0, 63247
Debugger [candidate] (63.927 ms) : 0, 63927
Remote Config [baseline] (632.814 µs) : 0, 633
Remote Config [candidate] (623.007 µs) : 0, 623
Telemetry [baseline] (10.81 ms) : 0, 10810
Telemetry [candidate] (11.42 ms) : 0, 11420
Flare Poller [baseline] (5.291 ms) : 0, 5291
Flare Poller [candidate] (5.301 ms) : 0, 5301
section iast
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (795.582 ms) : 0, 795582
BytebuddyAgent [candidate] (793.975 ms) : 0, 793975
AgentMeter [baseline] (11.305 ms) : 0, 11305
AgentMeter [candidate] (11.295 ms) : 0, 11295
GlobalTracer [baseline] (248.049 ms) : 0, 248049
GlobalTracer [candidate] (247.322 ms) : 0, 247322
IAST [baseline] (26.96 ms) : 0, 26960
IAST [candidate] (27.093 ms) : 0, 27093
AppSec [baseline] (32.947 ms) : 0, 32947
AppSec [candidate] (32.192 ms) : 0, 32192
Debugger [baseline] (65.705 ms) : 0, 65705
Debugger [candidate] (67.008 ms) : 0, 67008
Remote Config [baseline] (539.985 µs) : 0, 540
Remote Config [candidate] (535.624 µs) : 0, 536
Telemetry [baseline] (8.535 ms) : 0, 8535
Telemetry [candidate] (8.517 ms) : 0, 8517
Flare Poller [baseline] (3.451 ms) : 0, 3451
Flare Poller [candidate] (3.426 ms) : 0, 3426
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.073 s) : 0, 1072763
Total [baseline] (10.857 s) : 0, 10857190
Agent [candidate] (1.071 s) : 0, 1070515
Total [candidate] (10.843 s) : 0, 10842836
section appsec
Agent [baseline] (1.24 s) : 0, 1240019
Total [baseline] (11.001 s) : 0, 11001153
Agent [candidate] (1.247 s) : 0, 1247063
Total [candidate] (11.093 s) : 0, 11092576
section iast
Agent [baseline] (1.233 s) : 0, 1232679
Total [baseline] (11.264 s) : 0, 11264395
Agent [candidate] (1.234 s) : 0, 1234485
Total [candidate] (11.213 s) : 0, 11212586
section profiling
Agent [baseline] (1.193 s) : 0, 1192820
Total [baseline] (10.939 s) : 0, 10939474
Agent [candidate] (1.206 s) : 0, 1205788
Total [candidate] (11.102 s) : 0, 11101985
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.073 s -
Agent appsec 1.24 s 167.256 ms (15.6%)
Agent iast 1.233 s 159.916 ms (14.9%)
Agent profiling 1.193 s 120.057 ms (11.2%)
Total tracing 10.857 s -
Total appsec 11.001 s 143.963 ms (1.3%)
Total iast 11.264 s 407.205 ms (3.8%)
Total profiling 10.939 s 82.284 ms (0.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.071 s -
Agent appsec 1.247 s 176.549 ms (16.5%)
Agent iast 1.234 s 163.971 ms (15.3%)
Agent profiling 1.206 s 135.273 ms (12.6%)
Total tracing 10.843 s -
Total appsec 11.093 s 249.74 ms (2.3%)
Total iast 11.213 s 369.75 ms (3.4%)
Total profiling 11.102 s 259.149 ms (2.4%) 
gantt
    title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.204 ms) : 0, 1204
crashtracking [candidate] (1.199 ms) : 0, 1199
BytebuddyAgent [baseline] (632.47 ms) : 0, 632470
BytebuddyAgent [candidate] (632.185 ms) : 0, 632185
AgentMeter [baseline] (29.321 ms) : 0, 29321
AgentMeter [candidate] (29.205 ms) : 0, 29205
GlobalTracer [baseline] (259.002 ms) : 0, 259002
GlobalTracer [candidate] (258.783 ms) : 0, 258783
AppSec [baseline] (33.449 ms) : 0, 33449
AppSec [candidate] (33.379 ms) : 0, 33379
Debugger [baseline] (64.287 ms) : 0, 64287
Debugger [candidate] (65.305 ms) : 0, 65305
Remote Config [baseline] (632.504 µs) : 0, 633
Remote Config [candidate] (633.563 µs) : 0, 634
Telemetry [baseline] (10.092 ms) : 0, 10092
Telemetry [candidate] (9.126 ms) : 0, 9126
Flare Poller [baseline] (6.179 ms) : 0, 6179
Flare Poller [candidate] (4.535 ms) : 0, 4535
section appsec
crashtracking [baseline] (1.216 ms) : 0, 1216
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (659.137 ms) : 0, 659137
BytebuddyAgent [candidate] (663.932 ms) : 0, 663932
AgentMeter [baseline] (11.955 ms) : 0, 11955
AgentMeter [candidate] (12.092 ms) : 0, 12092
GlobalTracer [baseline] (258.275 ms) : 0, 258275
GlobalTracer [candidate] (259.725 ms) : 0, 259725
IAST [baseline] (25.477 ms) : 0, 25477
IAST [candidate] (25.381 ms) : 0, 25381
AppSec [baseline] (167.705 ms) : 0, 167705
AppSec [candidate] (167.966 ms) : 0, 167966
Debugger [baseline] (66.509 ms) : 0, 66509
Debugger [candidate] (66.834 ms) : 0, 66834
Remote Config [baseline] (654.903 µs) : 0, 655
Remote Config [candidate] (649.586 µs) : 0, 650
Telemetry [baseline] (9.514 ms) : 0, 9514
Telemetry [candidate] (9.474 ms) : 0, 9474
Flare Poller [baseline] (3.656 ms) : 0, 3656
Flare Poller [candidate] (3.692 ms) : 0, 3692
section iast
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.189 ms) : 0, 1189
BytebuddyAgent [baseline] (795.68 ms) : 0, 795680
BytebuddyAgent [candidate] (798.147 ms) : 0, 798147
AgentMeter [baseline] (11.281 ms) : 0, 11281
AgentMeter [candidate] (11.31 ms) : 0, 11310
GlobalTracer [baseline] (247.884 ms) : 0, 247884
GlobalTracer [candidate] (248.226 ms) : 0, 248226
IAST [baseline] (27.112 ms) : 0, 27112
IAST [candidate] (27.184 ms) : 0, 27184
AppSec [baseline] (33.202 ms) : 0, 33202
AppSec [candidate] (35.211 ms) : 0, 35211
Debugger [baseline] (67.654 ms) : 0, 67654
Debugger [candidate] (64.62 ms) : 0, 64620
Remote Config [baseline] (566.158 µs) : 0, 566
Remote Config [candidate] (534.236 µs) : 0, 534
Telemetry [baseline] (8.678 ms) : 0, 8678
Telemetry [candidate] (8.615 ms) : 0, 8615
Flare Poller [baseline] (3.483 ms) : 0, 3483
Flare Poller [candidate] (3.498 ms) : 0, 3498
section profiling
crashtracking [baseline] (1.202 ms) : 0, 1202
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (682.637 ms) : 0, 682637
BytebuddyAgent [candidate] (692.428 ms) : 0, 692428
AgentMeter [baseline] (8.554 ms) : 0, 8554
AgentMeter [candidate] (8.627 ms) : 0, 8627
GlobalTracer [baseline] (216.313 ms) : 0, 216313
GlobalTracer [candidate] (218.58 ms) : 0, 218580
AppSec [baseline] (32.645 ms) : 0, 32645
AppSec [candidate] (33.308 ms) : 0, 33308
Debugger [baseline] (67.337 ms) : 0, 67337
Debugger [candidate] (67.997 ms) : 0, 67997
Remote Config [baseline] (631.883 µs) : 0, 632
Remote Config [candidate] (630.575 µs) : 0, 631
Telemetry [baseline] (8.977 ms) : 0, 8977
Telemetry [candidate] (9.002 ms) : 0, 9002
Flare Poller [baseline] (3.739 ms) : 0, 3739
Flare Poller [candidate] (3.738 ms) : 0, 3738
ProfilingAgent [baseline] (100.117 ms) : 0, 100117
ProfilingAgent [candidate] (98.862 ms) : 0, 98862
Profiling [baseline] (100.694 ms) : 0, 100694
Profiling [candidate] (99.436 ms) : 0, 99436
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master charles.yu/djm-0000/fix-spark-plan-metrics
git_commit_date 1771530866 1771883443
git_commit_sha af8b844 89df516
release_version 1.60.0-SNAPSHOT~af8b84438c 1.60.0-SNAPSHOT~89df516e40
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1771886034 1771886034
ci_job_id 1449610690 1449610690
ci_pipeline_id 98402425 98402425
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-7oeezgiu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-7oeezgiu 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-178.164µs; -71.648µs] or [-6.224%; -2.503%]		 unsure
[-501.595µs; -101.424µs] or [-6.196%; -1.253%]		 unstable
[-84.186op/s; +210.373op/s] or [-6.680%; +16.694%]		 2.738ms 7.794ms 1323.281op/s 2.863ms 8.096ms 1260.188op/s
scenario:load:insecure-bank:iast:high_load better
[-232.201µs; -122.324µs] or [-9.249%; -4.872%]		 better
[-649.869µs; -196.348µs] or [-8.878%; -2.682%]		 unstable
[-51.865op/s; +280.802op/s] or [-3.681%; +19.927%]		 2.333ms 6.897ms 1523.625op/s 2.511ms 7.320ms 1409.156op/s
scenario:load:petclinic:tracing:high_load worse
[+0.646ms; +1.431ms] or [+3.736%; +8.275%]		 unsure
[+390.535µs; +1593.678µs] or [+1.369%; +5.585%]		 unstable
[-36.666op/s; +12.666op/s] or [-13.938%; +4.815%]		 18.325ms 29.525ms 251.062op/s 17.287ms 28.533ms 263.062op/s
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.309 ms) : 18120, 18498
.   : milestone, 18309,
appsec (18.697 ms) : 18507, 18887
.   : milestone, 18697,
code_origins (17.74 ms) : 17566, 17913
.   : milestone, 17740,
iast (17.572 ms) : 17397, 17746
.   : milestone, 17572,
profiling (18.868 ms) : 18676, 19061
.   : milestone, 18868,
tracing (17.735 ms) : 17560, 17910
.   : milestone, 17735,
section candidate
no_agent (19.149 ms) : 18959, 19339
.   : milestone, 19149,
appsec (18.765 ms) : 18575, 18956
.   : milestone, 18765,
code_origins (17.884 ms) : 17707, 18061
.   : milestone, 17884,
iast (17.693 ms) : 17520, 17865
.   : milestone, 17693,
profiling (18.581 ms) : 18394, 18769
.   : milestone, 18581,
tracing (18.584 ms) : 18400, 18768
.   : milestone, 18584,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.309 ms [18.12 ms, 18.498 ms] -
appsec 18.697 ms [18.507 ms, 18.887 ms] 387.655 µs (2.1%)
code_origins 17.74 ms [17.566 ms, 17.913 ms] -569.857 µs (-3.1%)
iast 17.572 ms [17.397 ms, 17.746 ms] -737.718 µs (-4.0%)
profiling 18.868 ms [18.676 ms, 19.061 ms] 558.805 µs (3.1%)
tracing 17.735 ms [17.56 ms, 17.91 ms] -574.104 µs (-3.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.149 ms [18.959 ms, 19.339 ms] -
appsec 18.765 ms [18.575 ms, 18.956 ms] -383.502 µs (-2.0%)
code_origins 17.884 ms [17.707 ms, 18.061 ms] -1.265 ms (-6.6%)
iast 17.693 ms [17.52 ms, 17.865 ms] -1.456 ms (-7.6%)
profiling 18.581 ms [18.394 ms, 18.769 ms] -567.545 µs (-3.0%)
tracing 18.584 ms [18.4 ms, 18.768 ms] -565.167 µs (-3.0%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.178 ms) : 1167, 1190
.   : milestone, 1178,
iast (3.247 ms) : 3201, 3294
.   : milestone, 3247,
iast_FULL (5.788 ms) : 5730, 5846
.   : milestone, 5788,
iast_GLOBAL (3.641 ms) : 3584, 3698
.   : milestone, 3641,
profiling (2.125 ms) : 2107, 2144
.   : milestone, 2125,
tracing (1.779 ms) : 1762, 1795
.   : milestone, 1779,
section candidate
no_agent (1.168 ms) : 1157, 1179
.   : milestone, 1168,
iast (2.998 ms) : 2962, 3033
.   : milestone, 2998,
iast_FULL (5.736 ms) : 5680, 5793
.   : milestone, 5736,
iast_GLOBAL (3.463 ms) : 3418, 3508
.   : milestone, 3463,
profiling (2.046 ms) : 2025, 2067
.   : milestone, 2046,
tracing (1.761 ms) : 1747, 1776
.   : milestone, 1761,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.178 ms [1.167 ms, 1.19 ms] -
iast 3.247 ms [3.201 ms, 3.294 ms] 2.069 ms (175.6%)
iast_FULL 5.788 ms [5.73 ms, 5.846 ms] 4.61 ms (391.3%)
iast_GLOBAL 3.641 ms [3.584 ms, 3.698 ms] 2.463 ms (209.0%)
profiling 2.125 ms [2.107 ms, 2.144 ms] 947.25 µs (80.4%)
tracing 1.779 ms [1.762 ms, 1.795 ms] 600.424 µs (51.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.168 ms [1.157 ms, 1.179 ms] -
iast 2.998 ms [2.962 ms, 3.033 ms] 1.83 ms (156.6%)
iast_FULL 5.736 ms [5.68 ms, 5.793 ms] 4.568 ms (391.1%)
iast_GLOBAL 3.463 ms [3.418 ms, 3.508 ms] 2.295 ms (196.5%)
profiling 2.046 ms [2.025 ms, 2.067 ms] 877.54 µs (75.1%)
tracing 1.761 ms [1.747 ms, 1.776 ms] 593.122 µs (50.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master charles.yu/djm-0000/fix-spark-plan-metrics
git_commit_date 1771530866 1771883443
git_commit_sha af8b844 89df516
release_version 1.60.0-SNAPSHOT~af8b84438c 1.60.0-SNAPSHOT~89df516e40
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1771885743 1771885743
ci_job_id 1449610691 1449610691
ci_pipeline_id 98402425 98402425
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-h7ofqtix 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-h7ofqtix 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (3.773 ms) : 3552, 3993
.   : milestone, 3773,
iast (2.239 ms) : 2170, 2308
.   : milestone, 2239,
iast_GLOBAL (2.288 ms) : 2219, 2358
.   : milestone, 2288,
profiling (2.088 ms) : 2032, 2144
.   : milestone, 2088,
tracing (2.054 ms) : 2000, 2107
.   : milestone, 2054,
section candidate
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (3.717 ms) : 3501, 3934
.   : milestone, 3717,
iast (2.241 ms) : 2172, 2310
.   : milestone, 2241,
iast_GLOBAL (2.298 ms) : 2228, 2368
.   : milestone, 2298,
profiling (2.492 ms) : 2330, 2655
.   : milestone, 2492,
tracing (2.059 ms) : 2005, 2112
.   : milestone, 2059,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 3.773 ms [3.552 ms, 3.993 ms] 2.299 ms (156.1%)
iast 2.239 ms [2.17 ms, 2.308 ms] 765.776 µs (52.0%)
iast_GLOBAL 2.288 ms [2.219 ms, 2.358 ms] 815.158 µs (55.3%)
profiling 2.088 ms [2.032 ms, 2.144 ms] 614.931 µs (41.7%)
tracing 2.054 ms [2.0 ms, 2.107 ms] 580.189 µs (39.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 3.717 ms [3.501 ms, 3.934 ms] 2.247 ms (152.8%)
iast 2.241 ms [2.172 ms, 2.31 ms] 770.259 µs (52.4%)
iast_GLOBAL 2.298 ms [2.228 ms, 2.368 ms] 827.492 µs (56.3%)
profiling 2.492 ms [2.33 ms, 2.655 ms] 1.022 ms (69.5%)
tracing 2.059 ms [2.005 ms, 2.112 ms] 588.088 µs (40.0%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~89df516e40, baseline=1.60.0-SNAPSHOT~af8b84438c
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.707 s) : 14707000, 14707000
.   : milestone, 14707000,
appsec (15.128 s) : 15128000, 15128000
.   : milestone, 15128000,
iast (18.34 s) : 18340000, 18340000
.   : milestone, 18340000,
iast_GLOBAL (17.907 s) : 17907000, 17907000
.   : milestone, 17907000,
profiling (14.774 s) : 14774000, 14774000
.   : milestone, 14774000,
tracing (14.675 s) : 14675000, 14675000
.   : milestone, 14675000,
section candidate
no_agent (15.355 s) : 15355000, 15355000
.   : milestone, 15355000,
appsec (14.995 s) : 14995000, 14995000
.   : milestone, 14995000,
iast (18.166 s) : 18166000, 18166000
.   : milestone, 18166000,
iast_GLOBAL (17.83 s) : 17830000, 17830000
.   : milestone, 17830000,
profiling (15.319 s) : 15319000, 15319000
.   : milestone, 15319000,
tracing (14.58 s) : 14580000, 14580000
.   : milestone, 14580000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.707 s [14.707 s, 14.707 s] -
appsec 15.128 s [15.128 s, 15.128 s] 421.0 ms (2.9%)
iast 18.34 s [18.34 s, 18.34 s] 3.633 s (24.7%)
iast_GLOBAL 17.907 s [17.907 s, 17.907 s] 3.2 s (21.8%)
profiling 14.774 s [14.774 s, 14.774 s] 67.0 ms (0.5%)
tracing 14.675 s [14.675 s, 14.675 s] -32.0 ms (-0.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.355 s [15.355 s, 15.355 s] -
appsec 14.995 s [14.995 s, 14.995 s] -360.0 ms (-2.3%)
iast 18.166 s [18.166 s, 18.166 s] 2.811 s (18.3%)
iast_GLOBAL 17.83 s [17.83 s, 17.83 s] 2.475 s (16.1%)
profiling 15.319 s [15.319 s, 15.319 s] -36.0 ms (-0.2%)
tracing 14.58 s [14.58 s, 14.58 s] -775.0 ms (-5.0%)

@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from 4e5bdc7 to ba09c80 Compare February 9, 2026 14:48
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch 5 times, most recently from cde7981 to e52fbc5 Compare February 19, 2026 21:41
@charlesmyu charlesmyu mentioned this pull request Feb 19, 2026
Draft
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from e52fbc5 to e413d1d Compare February 19, 2026 22:02
@charlesmyu charlesmyu added inst: apache spark Apache Spark instrumentation type: enhancement Enhancements and improvements labels Feb 19, 2026
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from e413d1d to 89df516 Compare February 23, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

inst: apache spark Apache Spark instrumentation type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@charlesmyu